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ABSTRACT 

The Educational Quality Indicators (EQl) initiative 
was a collaboration between Alberta Education (Alberta, Canada) and 
12 school jurisdictions to develop and implement indicator systems to 
measure the success of the educational enterprise. Ten concurrent 
collaborative projects were conducted between 1989 and 1992 to 
develop indicator systems for a broad range of outcomes, data 
collection and analysis methods, and ways to report and communicate 
the information to different audiences. An overview confirms that EQl 
has had a positive impact on education in Alberta. Alberta 
superintendents have adopted the idea of indicator systems. Many are 
using the EQl materials, and the provincial department of education 
is also incorporating ideas and strategies into its information 
system. The EQl initiative has demonstrated that a large-scale reform 
initiative focused on producing better information about education 
can help improve education. While there is no single indicator 
system, and none is a panacea, a useful indicator system can be 
constructed to focus on student outcomes through multiple measures. 
Three tables present information about the projects. An appendix 
gives the table of contents from the final report on the project, 
"Achieving Quali'tVt" which is published separately. (Contains 19 
references.) (SLD) 



* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
***************************^^ 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Reseaich and improvement 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 



This document has been reproduced as 
received from the person or organization 
originating it 
O Minor changes have been made to improve 
reproduction quality 

• Points of view or opinions stated in this docu 
ment do not necessarily represent oHiCiai 
OERi position or policy 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



00 



Lessons from the Educational Quality Indicators Initiative 



00 

e 



Nelly McEwen 
Policy and Planning 
Alberta Education 



Paper presented at the annual meeting of the Canadian Educational 
Researchers' Association, Ottawa, June 12, 1993 



2 

BEST COPY AVAILABLE 



Lessons from the Educational Quality Indicators Initiative 



The Educational Quality Indicators (EQI) initiative was a collaboration between Alberta 
Education and twelve school jurisdictions to develop and implement indicator systems 
to measure the success of the educational enterprise. Its primary goal was to assist the 
provincial government and school jurisdictions in providing better information to 
improve planning, policy and decision making. Towards this end, ten concurrent 
collaborative action research projects were initiated in 1989 and concluded in 1992. 
These projects developed indicator systems that include a broad range of outcomes, 
methods to collect, analyze and interpret data, and ways to report and communicate the 
information to different audiences. Researchers, practitioners and policy makers can 
learn a number of lessons from the EQI initiative. New knowledge has been generated 
in several areas of educational inquiry and in how we view learning and teaching. The 
second major focus discusses the implications of managing large-scale projects for 
optimum impact of an innovation. 



Educational Reform in Alberta 

The eighties saw a new wave of educational reform in Canada, the United States, and 
indeed around the world. Perceived shortcomings in education, and international 
competition in all sectors, served as a catalyst for the current accountability and reform 
movements. Within this context of global educational reform, Alberta introduced 
a number of initiatives to improve education in the province. This reform effort was 
stimulated by a series of changes in the political, economic and social climate of the 
province. In 1982 Alberta introduced the Achievement Testing Program at grades 3, 6, 
and 9, and in 1984 reinstated the Diploma Examinations Program at grade 12 as part of 
the requirements for high school graduation. In 1984, other reform initiatives included 
the School Act Review, the Management and Finance Plan, the Review of Secondary 
School Programs, and the introduction of five evaluation policies (student, teacher, 
program, school, and system). Two primary beliefs influenced the reform initiatives in 
Alberta. First, the student is the central reason for schooling, and second, education is a 
purposeful endeavor that can be assessed. To assist in ensuring that these beliefs became 
operational, Alberta Education promoted a results-based approach to education to provide 
a better balance among inputs, processes and results. 
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The EQI Initiative 

The Educational Quality Indicators (EQI) initiative was one of the projects designed 
to promote results-based education. Alberta Education committed $1 million to the 
EQI projects between 1989 and 1992. An important feature was that it introduced 
a collaborative model to the provincial reform initiatives. EQI participants became 
members of a provincial committee which discussed and recommended directions to 
Alberta Education in achieving results-based education. 

An indicator system is a tool to focus reform and improvement. The goals, or 
intended benefits of implementing indicator systems, are to assess the effectiveness 
and efficiency of the educational enterprise, to improve education, and to provide a 
mechanism for accountability. The major purpose of indicator systems is to provide 
ENHANCED information about education for IMPROVED planning, policy, practice, 
and decision making. EQI was designed to produce a set of indicators, appropriate 
standards, and accompanying methods to measure the performance of the educational 
enterprise. The proposed system of measuring success took into consideration and reflects 
government policy and the goals of schooling. It addressed two essential questions: 

1. Are students learning to their potential? 

2. Is the educational system supporting student learning effectively and 
efficiently? 



The key results expected over the three years of the EQI initiative were indicator 
systems that include a broad range of student outcomes, points of reference for comparing 
results, and an interpretative framework to describe variation among students and schools; 
methods to collect, analyze and interpret the data provided by the indicators; and ways to 
report and communicate the information to different audiences. A fundamental principle 
was that no single indicator, or even group of indicators, could fully describe the 
complexity of education. Any system would include many indicators, measured by 
both quantitative and qualitative methods, for selected dimensions. 

The conceptual framework for the EQI initiative guided the direction of the ten 
concurrent action research projects and the discussion of results-based education within 
Alberta Education. A four-dimensional model of education was developed which 
consists of partners (schooling, family and society), conditions (context, inputs and 
processes), student outcomes (cognitive, affective and behavioral) and time (grades 3, 6, 
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9 and 12) (McEwen & Zatko, 1989). The model draws on the work of Carroll (1963), 
Hymel (1988), Walberg (1984), Oakes (1986) and Shavelson etal (1987). The EQI 
model extends the work of the above by consolidating the joint responsibility of the 
partners who contribute specific conditions to developing student learning over time. 
Education is a complex social enterprise. Focusing on schooling alone, without regard 
for the other influences that shape its direction, will likely not result in significant change. 
This model incorporates the responsibilities of the other major partners and recognizes 
conditions which contribute to student outcomes. For educational constituencies to 
determine whether or not students are making appropriate progress, it is necessary to 
examine results over time to determine what changes should and have been effected. 

The EQI initiative set three criteria for a successful indicator system: an 
interpretative framework to describe variation among students and schools; student 
outcomes related to the educational enterprise; and points of reference for comparing 
results (Alberta Education, 1988). School jurisdictions that were already concerned 
with measuring educational quality were identified and invited to submit a proposal. 
The twelve school jurisdictions whose proposals were accepted represent almost half 
(48%) of the students in public and separate schools in the province. 

Alberta Education supported the EQI initiative by providing resources (funds and 
consultation), information and coordination. The information included an annotated 
bibliography containing 350 citations, a report on methodological considerations 
recommending a balanced approach between quantitative and qualitative methods, and a 
critique of more than 90 available assessment instruments (Alberta Education, 1990). 
Recommended implementation strategies to minimize the negative impacts of indicator 
systems included the use of multiple goals, indicators, methods, levels of analysis and 
participants. See McEwen and Zatko (1989) for a description of the conceptual 
framework and implement ion plan, McEwen and Chow (1991) for a discussion of 
issues in implementing indicator systems, and Alberta Education (1993) for the final 
report of the EQI initiative. 
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Educational Inquiry 

EQI established a conceptual framework for a major reform and improvement initiative, 
yet permitted flexibility within that framework to accommodate participants' areas of 
interest. It has set a new standard for how to go about research to get maximum benefit 
from the investment. We have learned a great deal about project diversity, a direction for 
improvement, a wide range of desired student outcomes, and conditions that enhance 
learning and promote system effectiveness and efficiency. 

The Projects - Strength in Diversity 

The projects represent a diversity of approaches. Each participating school jurisdiction 
was responsible for its own project because each represents a community which has 
values it wishes to impart to its students through its goals, priorities and expectations. It 
was important that each proje reflect these aspirations by involving its public - 
students, teachers, administrators, trustees, parents, and others - in discussions on which 
indicators to include and why. Without input from the larger community, the indicators 
might not provide a representative picture of the expected outcomes of the educational 
system. 

School jurisdictions, if they are to make sense of the outcomes of their educational 
enterprise, need to interpret the results in terms of their local circumstances. In the 
process of developing a preliminary set of indicators, trying it out in their own 
jurisdiction and refining it on the basis of the results, participating jurisdictions took 
the opportunity to develop and use indicators that are flexible, responsive to local needs, 
have community support, and can provide them with the type of information they need 
for setting policies, practices and procedures. 

Each of the ten projects focused on specific aspects of educational improvement and 
accountability. They can be grouped into three major areas - system accountability, 
teaching and learning, and alternative student assessment strategies. Three projects 
focused on accountability: the Lacombe/Rocky Mountain Project developed a 
comprehensive school system review process, Grande Prairie developed prototype 
school and district profiles, and Lethbridge Public developed a collaborative model for 
school and program evaluation. Two projects examined teaching and learning: Spirit 
River identified 26 behaviors of effective teaching and prepared a composite teacher 
performance baseline for the district; Fort McMurray Catholic implemented the principles 
of outcome-based education (OBE) in mathematics. Five projects examined alternative 
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ways to measure student success: Edmonton Public developed mechanisms for portfolio 
assessment and student growth, Calgary Public and Catholic developed indicators of 
quality student performance in art and mathematics, Brooks explored ways to measure 
responsible student behavior, Fort McMurray Public developed a three-level approach 
to measuring desirable student social outcomes, and Lethbridge Catholic developed a 
taxonomy of student affective behaviors. 

Achieving Quality, the final report of the EQI initiative, contains a summary of 
the initiative and each of the ten projects. A final chapter summarizes the major 
accomplishments, proposes potential provincial indicators, and suggests further steps in 
expanding the use of indicators in school jurisdictions. The report was released at the 
January 1993 EQI conference organized by Alberta's association of school superintendents 
and attended by over 300 people. Unless otherwise noted, all project references are to this 
publication. The table of contents is in the appendix. 

A Direction for Improvement 

The indicator systems developed by the participating school jurisdictions are based on 
theory and practice. An extensive review of the literature during the developmental phase 
ensured that teams were aware of conceptual frameworks and incorporated these into 
developing their indicator systems. The field tests tempered theory with practical 
realities to ensure that indicators that survived this phase were practical and had direct 
application for school districts. This direction for improvement incorporated a focus on 
student outcomes based on the goals of schooling, a balance among cognitive, affective 
and behavioral outcomes, multiple ways to collect and report information, and the active 
involvement of educational partners (students, staff, parents, stakeholders, and the 
public). 

The local indicator systems have support within their communities because of 
the active involvement of partners in their development. Several models for assessing 
the effectiveness and efficiency of instructional programs, delivery systems, and overall 
system performance were developed. In addition, several innovative alternative student 
assessment strategies were developed. They are valuable additions to the existing set of 
measurement instruments because they are based on theory and refined through practice. 
Other districts can use these products to assess a broader range of outcomes. These 
products can now be adapted by other districts and used to suit their local purposes. 
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Student Outcomes 

An important aspect of the EQI initiative was to address the question of student learning 
potential. This focuses on two dimensions of the EQI model - student outcomes 
(cognitive, affective, behavioral) and time (grades 3, 6, 9, 12). Traditionally, reported 
student outcomes have tended to focus almost exclusively on the cognitive domain. If 
the purpose of education is to develop the whole child, then it is important to address all 
domains: cognitive, affective, physical and social. New knowledge generated about 
student outcomes in each domain is discussed in turn. 

Cognitive Domain - Achievement was an important indicator of cognitive outcomes in 
many of the projects. Five (Lacombe/Rocky Mountain, Grande Prairie, Spirit River, Fort 
McMurray Catholic, and Brooks) reported student achievement measured by Alberta's 
achievement tests at grades 3, 6, and 9, and the diploma exams at grade 12. These tests 
are important because they assess the four core subjects - English, social studies, 
mathematics, science - and are administered annually to students at the four grade levels, 
thereby providing a provincial picture of achievement and a basis for comparing local 
results with aggregated provincial results. In addition to the usual descriptive statistics, 
annual provincial reports indicate the percentage of students achieving the acceptable 
standard (50% or higher) and the standard of excellence (80% or higher). High school 
students who achieve an average of 80% or higher in designated subjects qualify for a 
Rutherford Scholarship if they pursue post- secondary studies. This award is a powerful 
incentive for secondary students to excel in school. 

Performance assessment was an alternative approach to paper-and-pencil measures 
in two of the projects which focused on providing evidence of student learning and 
achievement. Edmonton Public was interested in documenting student growth over time 
on six essential learning outcomes (communication, responsible citizenship, well-being, 
knowledge, inquiry, aesthetic appreciation). It developed 15 tasks which incorporated 
these outcomes and measured student growth on them. After two years in school, 204 
students in nine pilot schools ranged in performance on 13 indicators (such as reading, 
making choices, making decisions) from grade 1 to grade 3. About three quarters of the 
students achieved at or above the grade 1 level after one year, and about nine in ten 
achieved at or above the grade 2 level after two years in school. However, a quarter of 
the students were not yet reading or writing at the grade 2 level after two years in school 
(Edmonton, p. 94). The Calgary project examined the product of student learning in art 
and the process of student thinking in mathematical problem solving. Calgary defined 
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quality indicators as observable characteristics of excellence established through 
consensus of professional judgment among practicing teachers. It developed two 
handbooks of exemplars of student performance for elementary, junior and senior high 
school - the Educational Quality Indicators Art Handbook and the Problem Solving 
Profiles. Both contain the quality indicators for the respective subjects, student 
exemplars, and expert observation of the student performance. 

Two projects dealt specifically with mathematics (Fort McMurray Catholic and 
Calgary) and two with art (Calgary and Lethbridge Public). The approaches, however, 
dealt with achievement quite differently. While Fort McMurray's OBE project required 
80% mastery in mathematics on units before a student could proceed, Calgary was 
interested in problem solving ability and rated the process of student thinking, as 
demonstrated by the solution, according to four categories: preliminary, partial, complete, 
elegant. Both projects were interested in students' perceptions as well as achievement. 
Fort McMurray used the School Subjects Attitude Scales (Nyberg & Clarke, 1983), a 
semantic differential to measure students' attitudes toward mathematics on three factors: 
evaluation, usefulness and difficulty. Calgary asked students to describe the problem 
(boring, challenging, confusing, fun) and to indicate how they felt after trying to solve it 
(discouraged, satisfied, frustrated, happy). This additional information helped teachers 
improve the processes provided in the classroom. 

Art achievement was also assessed differently. The Lethbridge Public team 
developed a collaborative model for school and program evaluation based on a review 
of its former model and revised on the basis of evaluating its art and library programs. 
Student achievement in art was assessed using diagnostic tests which helped teachers 
determine student knowledge and performance. Participation/effort and the art project 
evaluation were assessed according to impressionistic criteria. In addition, student, 
teacher, administrator, and parents' perceptions about the art program provided multiple 
perspectives on the program. Calgary, on the other hand, was interested in student 
performance in art. Art was described by three indicators of quality: relationships, 
handling and meaning. Exemplars of performance at four levels are presented in the 
EQI-Art handbook, which contains examples of student artwork resulting from a common 
lesson. Each example is accompanied by student intention regarding the artwork, and an 
expert's observations. These exemplars demonstrate the diversity of student performance 
given a common stimulus and illustrate how student creativity can be documented and 
modeled for others. Calgary's emphasis on student intention adds an important dimension 
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to assessment because it demonstrates how students perceive their work and helps in 
interpreting it. Calgary concluded that quality art assessment is a shared experience 
between students and teachers which is encouraged by shared exemplars and vocabulary. 



Affective Domain - Lethbridge Catholic defined the affective domain as that area of 
student development which includes attitudes, interests, appreciation, feelings, emotions 
and predispositions. It developed a model of affective learning and a taxonomy of 
student behaviors in five areas: self- worth, interpersonal relationships, world awareness, 
learning, and spiritual life. Each of these five indicators is expressed and observed 
through three behaviors which are dynamic and constructive, and can be observed and 
celebrated as signs of learning and growth. A Celebration of Learning contains three 
observable behaviors for each of the five indicators and seven typical actions that 
represent each behavior. For example: 

(Indicator) LEARNING 

(Behavior) 10. Shows a desire to learn and to search for truth by being open and 
receptive to learning, affirming it and reflecting it with happiness, enthusiasm and 
satisfaction. 
(Typical Actions) 

1 . Is ready to learn at school or at home. 

2. Is eager and enthusiastic to learn. 

3. Comes to school happily and willingly. 

4. Makes conscious choices to learn. 

5. Learns from others. 

6. Shows interest in learning. 

7. Tries to verify. 

In total, there are 15 behaviors and 105 typical actions. The intent of this taxonomy 
and its accompanying instruments is to nurture desirable behavior. The instruments include 
a class checklist to allow for a quick method of recording reminders of observations, and 
self-evaluations for students (separate versions for primary (grades 1-3), elementary (grades 
4-6) and secondary (grades 7-12). The results of the project showed that affective learning 
was addressed by describing behaviors which are desirable; affirming students when 
desirable behavior J are observed; conferring with students and parents regarding the absence 
of specific behaviors; and teaching, modeling, and valuing behaviors identified as indicators 
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of affective learning (Lethbridge RCSSD No. 9, 1993, p. 20). The authors recommended 
that Alberta Education adopt the definition of a student "working to potential" as: 
a student who keeps trying and does not give up. Potential is a dynamic, 
continuing condition not a state to be realized. It follows that the criterion for 
checking any feedback on student performance is whether it allows and encourages 
students to keep trying. The skill of making choices becomes central for students 
(ibid, p. 26). 

This project provides a valuable model for the affective domain that can be used by 
students, teachers and parents to help students develop positive behavior. Cooperation 
between the home and school is seen as essential in helping students develop their 
potential. The materials and procedures are intended for classroom use and were designed 
to be simple, defensible, observable, time efficient and useful. The descriptors allow 
teachers to sharpen their perceptions and base their instructional and reporting decisions 
on direct observation of behavior which is dynamic, active and vital. Observations of 
affective learning are noted, reported and celebrated. What has not yet been learned is 
identified, and what can be done about it is discussed with the student, parent and teacher 
(ibid). 

Social Domain - Fort McMurray Public developed a model of social competence. It used 
a systems perspective based on a modified CIPP model as the conceptual framework. 
The team then cross-referenced a number of existing assessment instruments to the 
desirable social outcomes: behaviors, perceptions, and social appropriateness. These 
instruments were then field-tested in selected classrooms to confirm the quality of data 
gathered and to review the ease and practicality of instrument administration. The results 
of the field test indicated that teachers were uncomfortable in administering the tests, and 
in analyzing the results. The instruments were considered "impractical" for use with 
whole classes because of the amount of time taken to administer them, and the low return 
of parents' ratings of their children on the instruments. The project team subsequently 
developed a three-level approach to measuring social skills. The first level was teacher 
reports to parents, at report card time, of the social development of their children. Each 
teacher rated students as being well developed, developing appropriately, or needing 
further development. The ratings were based on personal factors, interpersonal factors, 
social reasoning skills, and classroom behaviors. If any of the students, parents, or 
teachers wished or required a more detailed assessment (level two), the district's Social 
Skills Diagnostic Screen (developed for the EQI project) would be completed by the 
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teacher for use in providing a rating for the report card, forming the basis for discussion 
with parents and/or the student, or for developing a moderate program of remedial action. 
A form could also be completed by the student, the parents, and a composite student 
profile compiled for discussion. If detailed knowledge of the student's social development 
was required (level three), the student would be referred to a counselor for further 
assessment and the development of an Individualized Program Plan. This practical 
approach meant that for the majority of students, the teacher was able to assess social 
development and report it to parents. The district's report cards were revised to 
incorporate the new approach. 

The Social Skills Diagnostic Screen (SSDS) consists of fcur sections representing 
investigations of the same four areas identified at level one. Each section contains a series 
of simple to administer and score questions. In total, 44 questions are rated on a five- 
point scale (rarely, occasionally, about half time, often, almost always). The screen may 
be administered in either individual or group settings, and may be completed by the 
student, a student's peer, the teacher, a parent, or any other individual known to the subject 
in other than a casual manner. The screen is designed for school-aged subjects across all 
grade levels. Administration time varies from 10 to 15 minutes with older subjects to 20 
to 30 minutes with younger subjects who need to be interviewed on an individual basis 
(Fort McMurray Public, p. 128). The SSDS was field-tested to establish its validity and 
reliability. The screen's practicality, ease of administration, and guidelines for interpreting 
results make it attractive for districts that want to assess students' social skills. In addition 
to an administration manual, the project produced an activity manual to help teachers 
develop students' social skills. These materials are welcome additions to the existing 
instruments available for measuring and developing social skills. Social expectations 
and outcomes have been integrated into the district's policy and report cards, and became 
a priority in the 1992-93 school year. 

Behavioral Domain - Brooks developed a set of indicators of responsible student behavior 
in schools: high expectations; respect and recognition; pride in self, others, home, school 
and community; courteous, supportive, caring and sharing attributes; and mutual trust and 
communication. In the last year of the project, Brooks involved its community in 
affirming and supporting student behavior. These Character in Our Community meetings 
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led to a set of beliefs. The community defined character as: 

the ability to identify the elements of a circumstance or situation, to determine 
one's most constructive role in it, to carry out that role directly, to sustain it as 
long as necessary, and to learn from the consequences of one's actions ( Brooks, 
p. 119). 

A set of 16 belief statements followed this definition. For example: 

1. Good character is respecting yourself and others. No one has the right to 
take (without consent), harm or destroy another person's physical or mental 
well-being. No one has the right to devalue the character of another. 
3. The difference between rights, responsibilities, and privileges must be 
recognized. 

10. Community contributions are made by those who wish to institute improvement. 
12. An individual is accountable for personal actions and must live with the 
consequences (ibid.) 

Behavior also includes the physical domain. The Lacombe/Rocky Mountain project 
included physical well-being as part of personal learning, and used the Canada Fitness 
Award (Fitness Canada, 1984) and two locally developed measures, the Adolescent Health 
Questionnaire and selected questions on the Satisfaction Survey, to assess physical well- 
being. Grande Prairie also incorporated the Canada Fitness Award into its set of measures 
to assess student outcomes. 

Potential Provincial Indicators 

A major purpose of the EQI initiative was to develop a set of indicators for a provincial 
indicator system. At the fifth provincial meeting in January 1992, EQI partners discussed 
potential provincial indicators, effective practices, follow-up plans and issues. 
Participants recommended that provincial indicators should: 

• focus on student outcomes 

• be based on the goals of schooling and desirable personal characteristics 

• represent a balance among cognitive, affective and behavioral outcomes 

• provide corollary evidence (tests, surveys, interviews, records) 

• provide multiple perspectives (students, staff, parents, stakeholders, the public) 

The student indicators for which there was consensus include achievement, 
participation, creativity, graduation, attitudes, self-esteem, satisfaction, behavior, 
responsibility, and fitness. Each indicator is tied to one or more of Alberta's goals 
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of schooling and the desirable personal characteristics. Multiple measures and sources 
are recommended for each student outcome. This broader range of student outcomes 
contributes substantially to the existing information on students and complements the 
currently available information on context, inputs/resources and outcomes. Table 1 
presents the indicators, measures, and sources for student outcomes for which there 
was consensus to include in a provincial indicator system. 

The proposed indicators can serve both provincial and local needs. Alberta's 
achievement testing and diploma examination programs provide districts with annual 
results for their students. Adding indicators of participation and completion provides 
a more comprehensive picture of student success. Whereas achievement provides a 
measure of academic excellence, participation provides an indication of equity. Together 
a better picture of performance results. Moreover, exemplary products by students kept 
in a portfolio or on a videotape provides evidence of the diversity of student performance. 
The proposed indicators can, therefore, not only be aggregated and reported at the 
provincial level, but also at the district and school levels. Those involving student 
diversity and nonacademic areas can be collected and reported locally. Jurisdictions 
are free to tailor information reports to their publics based on local priorities and 
expectations. Therefore, accountability can be served both provincially and locally, 
while respecting local autonomy. 

Measuring indicators in multiple ways gives districts confidence in the information 
provided by a variety of sources. Provincial sources provide common indicators which 
ensure comparability among districts and provide an overall picture for the province. For 
example, when achievement is measured by provincial achievement tests and diploma 
exams, Rutherford Scholarships, and performance assessments, districts can see how well 
students at different levels (grades 3, 6, 9, 12) compare to provincial standards. Trends 
over time indicate whether improvement is occurring. 

Many of the products can be used by other districts immediately. For example, the 
Social Skills Diagnostic Screen can measure student social skills and A Celebration of 
Learning describes student affective behavior. Schools are microcosms of society, and 
social and affective skills are important areas for student growth and development. 
Moreover, Calgary's EQI-Art Handbook and the Problem Solving Profiles provide 
exemplars of student performance and expert analysis that can help teachers identify 
quality work and students produce it. 
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Table 1 

Potential Provincial Indicators 



Goals 1 Indicator 



Measures 



Source(s) 



ERLC 



Students: 

1, 2, 3 achievement 



1,2 



participation 



1,2,7 creativity 



1-7 



1,2 



1-7 



4-7 



5,6 



graduation 



attitudes 



self-esteem 



satisfaction 



behavior 



responsibility 



fitness 



grades 3, 6, 9 achievement tests 
grade 12 diploma examinations 
Rutherford Scholarships 
performance assessment 

percentage of students taking 
courses and writing tests/exams 

evidence of exemplary work kept in 
portfolios (student products, videos) 

percentage of students who graduate 
from high school 

School Subjects Attitude Scales ^ 
other attitude surveys 

available instruments 3 
student surveys 
observation 

satisfaction surveys 
personal interviews 

A Celebration of Learning 4 
Social Skills Diagnostic Screen 5 

homework completion 

attendance 

vandalism 

Canada Fitness Award 



students 

district 

students 

district 

students 

students 

teachers, parents 
students, teachers, public 

students, teachers, parents 



students 

district 

district 

students 



r 



Goals of schooling: 1 communication, 2 content, 3 skills, 4 well-being, 5 citizenship, 6 world of work, 
7 desirable personal characteristics. Goals 1 and 3 are cognitive in nature, goal 2 contains both cognitive 
and affective aspects, and goals 4 to 7 essentially encompass all three domains. 



2 Students' attitudes toward subjects are assessed on three factors: evaluation, usefulness, difficulty. 

3 There are several instruments available. Consult the EQI Inventory of Assessment Instruments. 

4 Student behaviors are described in five groups which show learning in: self-worth, relating to others, 
world awareness, learning, spiritual life. See the Lethbridge Catholic report. 

^ Student behaviors are categorized into four areas: personal factors, interpersonal factors, social 

reasoning skills, classroom behaviors. See the Fort McMurray Public report- 
Source: Alberta Education, Achieving Quality, 1993, p. 151. 
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System Effectiveness and Efficiency 

EQI's second major question asked if the educational system is supporting student learning 
effectively and efficiently. This question relates primarily to the partners (schooling, 
family, society) and conditions (context, inputs, processes) of the conceptual framework. 
While all projects dealt with this through the type of support they provided in helping 
students achieve desired learning outcomes, three projects focused on system 
accountability, and two on teaching and learning. 

The Lacombe/Rocky Mountain project developed a comprehensive system review 
incorporating student, teacher, program, school and system evaluations. Each type of 
student outcome was tied to a particular goal of student learning and then indicators, 
measures, comparators and grades identified for each. An example follows. 



Intellectual Learning: Develop knowledge, skills and positive attitudes in language and communication. 



Outcomes 


Indicators 


Measures 


Comparators 


Grades 


cognitive 


achievement 


language arts 
achievement test 


provincial and 
system results 


3,6,9 






English 30 and 33 
diploma exams 


provincial and 
system results 


12 


affective 


attitude 


School Subjects 
Attitude Scales 


test norms 


6,9,12 


behavioral 


participation in 
achievement tests 
and diploma exams 

etc 


document review 


provincial rates 


3,6, 9,12 



Source: Lacombe/Rocky Mountain Project, Achieving Quality, 1993, p. 17. 

Conditions and processes included access to schooling, press for achievement, and 
professional teaching conditions. The project team identified indicators and measures for 
each area (home, classroom, school and system) for the three conditions. Table 2 presents 
examples from the full set. The comprehensive nature of the system review provides a 
model for other districts wishing to follow this approach. The 1991-92 results serve as a 
benchmark for subsequent reviews at five-year intervals with the intervening four years 
focusing on implementing needed improvements. 

O 
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Table 2 



Examples of the Lacombe/Rocky Mountain Indicators of Conditions 
and Processes Affecting Student Achievement 



Condition 



Area 



Indicator 



Measure 



access to schooling home 



press for achievement home 



family structure 
socioeconomic status 
student mobility 

attitudes and actions of 
importance to education 



survey, document review 



survey 



access to learning 



classroom 



press for achievement classroom 



instructional presentation 
etc 

teacher expectations 
etc 



self-evaluation, observation, 
survey 

self-evaluation, observation, 
survey 



access to schooling school 
review 

press for achievement school 



professional teaching school 
conditions 



location and size 
etc 



leadership 
etc 



class size 
etc 



document review, survey, 
committee study, external 



School Effectiveness Opinionnaire, 
external validation, survey 

document review, survey, 
School Effectiveness Opinionnaire 



access to schooling school system 



press for achievement school system 



professional teacning school system 
conditions 



location, size, assessment document review, survey 

base 
etc 

mission, goals, and actions document review, survey 

for improvement 
etc 

staff involvement in system document review, survey 

decision making 
etc 



Source: Lacombe/Rocky Mountain Project, Achieving Quality, 1993, p. 19. 
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Grande Prairie Public developed prototype school and district profiles to inform 
stakeholders of the quality of education in its district. The profiles provide information in 
four areas - student achievement, school climate, quality of instruction, and funding - and 
will be published annually. The quintessential objective of the school profile is to provide 
stakeholders with information that is meaningful and useful to them in planning school 
improvement and measuring the health of their schools (Grande Prairie, p. 40). Beginning 
with the 1992-93 school year, all schools in the district will produce annual profiles. 
These are an important mechanism for focusing improvement efforts and informing 
stakeholders about the quality of education in the district. 

Lethbridge Public developed a collaborative model for school and program evaluation. 
This district concluded that limited changes were realized by predominantly external 
evaluations because teachers and school-based personnel were not actively involved and 
hence gained little from the experience. A study of the related literature and procedures 
elsewhere led to the development of a collaborative model to empower school staffs to be 
actively and professionally involved in their school and program evaluations. The use of 
educational quality indicators served as a key component of the model (Lethbridge Public, 
p. 43). The district is continuing to expand the use of this model. 

Spirit River developed a collaborative teacher evaluation model. It identified 26 
behaviors of effective teaching and grouped them into four areas - planning and preparation, 
instructional strategies, communication skills, and pupil/teacher relationships. All teachers 
in the district took part in reviewing the set of 26 teaching behaviors and discussing teacher 
evaluation. Parents and students were surveyed on the importance of these behaviors from 
1990 to 1992. In 1991, teachers were observed on these behaviors and an aggregated 
composite performance baseline prepared. Behaviors receiving the lowest ratings became 
the target for professional development activities. Data collection was repeated in 1992 to 
provide a basis for comparison. Using the same criteria for both formative and summative 
evaluations has heightened teacher awareness about professional expectations. Student 
attitudes, achievement and discipline data were also collected. The district is using the 
data to determine the effect of teaching on student outcomes. 
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Fort McMurray Catholic implemented the principles of outcome-based education 
(OBE) in mathematics from grades 2 to 10. The premises of OBE are that all students 
can learn and succeed, success breeds success, and that schools control the conditions of 
success. The principles include designing down from significant outcomes, maintaining 
a focus on these outcomes, holding high expectations for all students, and providing 
expanded opportunities for learning. Quantitative and qualitative evidence indicated that 
the OBE process produced positive results in student achievement, responsibility, and 
attitudes in mathematics. The district is committed to the continuation and improvement 
of the OBE process. 

Synopsis 

This section has described major lessons about educational inquiry from the EQI 
initiative. Each project has contributed to our understanding of important student 
outcomes of schooling. If schooling is a purposeful activity whose goal is to foster 
human growth and potential, it behooves us to identify the major outcomes 
and subsequently find ways to measure whether progress is being made. Broadening 
desired student outcomes to include the affective, social, and behavioral domains from 
the currently emphasized cognitive domain will help to redress the imbalance that has led 
to some of the criticism of our schools. Society invests in public education because it is 
one of the best ways to promote the social and economic well-being of its citizens. These 
"investors" consequently have a right to know how well the investment is contributing to 
their welfare. 

Education is a shared responsibility among its major partners - the family, schooling, 
and society. Collaboration among the partners helps to ensure that our young people 
develop the knowledge, skills and attitudes we consider important. Partnerships such as the 
EQI initiative help us come to a common understanding of what is important in education, 
and how to go about determining if what we are doing is, in fact, having the desired effect. 
Assessment can take many forms. Alternative strategies that incorporate observation and 
professional judgment about growth and excellence can increase pedagogical validity in 
assessment and help us improve the quality of education. 
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Collaboration in Action 

The EQI initiative has not only generated new knowledge about student learning and 
support, but also about how to introduce innovation into the educational system for 
maximum impact. We learned a lot about leadership and scope, collaboration, 
communication, resources, and celebration. 

Leadership and Scope 

Alberta Education provided leadership in initiating EQI and in inviting leading districts 
to participate. Leadership was subsequently shared among the participants. Strong 
leadership was required from all the partners - Alberta Education, coordinators and their 
teams, and superintendents. Local leadership was essential for completing the individual 
projects. 

The scope of EQI was an important feature in its success. The conceptual framework 
is broad and allowed districts to choose aspects that interested them. While each project 
individually contributed to our understanding about important outcomes of schooling and 
conditions that enhance student learning, none alone is as powerful as all of them together. 
Each project also added to our appreciation of the complexity of the issues involved in 
defining important outcomes, measuring them, and informing different audiences about 
results. Since each project represents a particular community, the commonalities become 
the basis for collective action while unique interests remain local prerogatives. 

Districts are the natural partners with the provincial government to introduce change 
because they are empowered to deliver education to students. Since districts can act 
quickly to make changes in their schools, adoption is faster, more effective and more 
efficient than if an external agency had been engaged to develop an indicator system 
which the province could subsequently adopt, adapt or ignore. By collaborating with 
districts, the change agents are already in place and assume responsibility for adoption. 
Since the districts have been actively involved, they have a vested interest in the 
outcomes and are proceeding accordingly. 

Collaboration 

Collaboration was the modus operandi of the EQI initiative whose motto was "Collaboration 
in action". This motto, together with the logo, a stylized wild rose whose petals represent 
each of the ten projects, graces the cover of all final EQI reports. Alberta Education is at the 
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centre because its coordinating function touched all ten projects concurrently. Symbols 
represent the essence of what is important and the relationships among parts. 

EQI is the single largest cooperative project in Alberta dealing with educational 
indicators. A 1991 survey of project teams indicated that 17,000 Albertans were directly 
involved in the development and field testing of the indicator systems. Two thirds of the 
participants were students and a quarter were members of the community - parents, 
stakeholders, the public. The community was involved in providing perceptions of what 
is important in education. Participation (1,300 teachers and more than 4,000 members of 
the community) is an important ingredient in the successful implementation of an 
indicator system. The EQI initiative has become a model for collaboration between 
government and school jurisdictions. 



The EQI initiative was successful because it gave participants a chance to explore 
areas they considered important. While there is general agreement that better information 
can help to improve education beca : st it focuses attention on the significant outcomes of 
schooling, there is considerable diversity in thinking about what these outcomes are and 
how to go about determining whether they have been achieved. By giving participants the 
opportunity to explore what they thought was worthwhile, in cooperation with their local 
communities, the end result is a consensus of what is generally important. 

A community shares values, goals, priorities and expectations. Four of the projects 
took place in two cities - Lethbridge and Fort McMurray. Despi*^ this common 
community, each participating district selected aspects which intc sted it. T n Fort 
McMurray this ranged from social skills (public district) to matheii-ancs achievement, 
responsibility and attitudes (Catholic district). In Lethbridge, the Catholic district 
examined the affective domain while the public district developed school and program 
evaluations. Although both metropolitan projects focused on performance assessment, 
Calgary Public and Catholic concentrated on exemplars while Edmonton Public 
examined student growth. 
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This opportunity to pursue district interests resulted in local ownership and 
commitment to individual projects, with acceptance and appreciation of the areas 
explored by the other projects. The synergy among the partners helped to affirm 
the collective work of the initiative. 
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Communication 

Communication was an essential ingredient for successful integration of the projects. 
It took many forms and helped to promote the partnership among the participating 
jurisdictions. 

Internal Communication - EQI partners met regularly to exchange ideas, trends, issues 
and techniques. These semiannual meetings served to maintain personal contact among 
the key participants - project coordinators, superintendents, and Alberta Education staff - 
and to share information, discuss and interpret findings, and identify issues and strategies. 
Participants were able to discuss concerns in a safe, supportive environment where people 
offered constructive suggestions rather criticized the work. This setting encouraged 
members to share experiences in an honest and forthright manner. The meetings also 
established milestones that motivated people to meet deadlines. 

A local team was responsible for each project. Teams met regularly and selected 
members to provide alternative perspectives to help in ensuring a broad consensus for 
desired outcomes. Some of the projects expanded team membership to people in the 
community. 

The Collaborator was a quarterly newsletter introduced as a communications 
mechanism for the project teams and Alberta Education, and was available to others upon 
request. It informed readers of important events and provided an outlet for coordinators 
to contribute articles about their projects. Each June issue contained an annual status 
report. The Collaborator also provides a record of the EQI initiative. 

External Communication - It was also important to inform others about developments. 
Project teams routinely communicated with their boards about progress, and presented 
workshops and sessions to colleagues, associations and other organizations through 
meetings, conventions and conferences. Alberta Education also informed others of 
the initiative through its newsletter infocus, distribution of materials to educational 
organizations in Alberta and across Canada, and participation in local, provincial, 
national and international meetings and conferences. 
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Five teams participated in a symposium at the 1990 meeting of the Canadian Society 
for the Study of Education in Victoria and seven teams took part in the 1992 meeting in 
Charlottetown. The proceedings of this second symposium have been published in the 
June 1993 issue of the Alberta Journal of Educational Research. In addition, requests for 
information about EQI have come from governments, school jurisdictions, universities and 
other organizations across Canada, the United States, Europe, Asia and Australia. 

Celebrating Success - The formal conclusion of the initiative was celebrated at the 
January 1993 EQI Conference. Organized by the Conference of Alberta School 
Superintendents (CASS), the conference provided an opportunity for the ten project 
teams to present the materials they developed and the results achieved over the past 
three years. It was the first time that Alberta Education held a conference on a research 
endeavor, and the first time that CASS organized such an event for an Alberta Education 
initiative. The conference attracted Alberta superintendents, trustees, directors, 
departmental staff, and school district and government officials from other provinces. 
The opportunity for delegates to discuss findings with the project teams firsthand 
contributed substantially to the interest in the projects. In addition to showcasing the 
EQI projects, it provided a powerful incentive for the project teams to complete their 
reports because summaries were published in Achieving Quality, which was released at 
the conference. This report attracted much wider interest than had it been distributed 
solely through our usual release procedures. 

Celebrating the completion of an initiative this way is a major recommendation. 
Project teams worked extremely hard and deserved the accolades of their peers on a job 
well done. It also set up the communication and dissemination phase in a positive 
manner that encourages others to try the fruits of the labor. The conference marked 
the formal conclusion of the initiative, but the beginning of a new era of cooperation 
among educational partners interested in improving education. 



Resources 

No discussion of research, development and reform is complete without addressing the 
resource issues. The major resources are staff, dollars, information, professional 
development, support and time. Anyone wishing to embark on a major initiative should 
marshal the necessary resources for an extended period of time to ensure success. 
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Staff- Participants who are interested in and committed to a project and have the necessary 
expertise should be invited to volunteer. Lack of involvement in and commitment to such an 
undertaking may lead to attempts to thwart a project. Therefore, teams should be selected 
carefully to ensure project viability. Staff turnover is inevitable. It is imperative that 
provision be made for staff transitions. When there are several people working on a project, 
the disruption is minimized because another can assume leadership of the group. However, 
changes in leadership usually result in changes in emphases and procedures. This is 
particularly the case when the superintendent changes. 

Community involvement is also recommended. Without shared expectations and 
the support of the community in the determination of goals, priorities and expectations 
for its students, a local school jurisdiction may not address the questions its constituents 
want answered. All EQI projects involved their communities to enhance credibility. 

Information - This is an essential resource for informed discussion. Alberta Education 
provided technical documents to support the EQI initiative. It produced an annotated 
bibliography, a methodological report, and an inventory of assessment instruments. 
Furthermore, it provided professional development activities on these documents at the 
semiannual meetings. Considerable time was also spent on how to communicate results 
to diverse publics. Updated information was also provided. 

Support - The investment of $1 million in the ten projects provided an incentive to get 
school jurisdictions involved in promoting results-based education. The participating 
districts contributed resources in kind. Provincial coordination and support of the EQI 
initiative were important to keep the projects - rich in diversity of goals, outcomes, and 
approaches ~ focused on the desired result - better information to improve the quality 
of education. An external review at the midpoint provided another perspective on the 
emerging indicators and approaches. 

Local support was also important. Superintendents who were actively involved 
in the project ensured that the team had the resources required to complete the tasks 
effectively and efficiently. Where the superintendent played a less visible part, tne 
project had a lower profile in the district. 
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Time - The importance of time cannot be underestimated. The EQI initiative is now five 
years old. It took a year to develop the conceptual framework, implementation plan and 
the Terms of Reference, and to initiate the projects and organize the infrastructure. 
The projects were given three years of funding because experience with projects 
of this magnitude indicates that it takes about this much time to accomplish desired 
results barring undue changes in staff. Teams spent the first year developing their 
indicator systems, a year to field test their prototypes and a year to refine and complete 
their projects. Despite changes in project coordinators, superintendents, and three years 
of change in Alberta, the initiative met its target and was completed on time. This fifth 
year is one of completing the documentation and promoting the results. 

Invariably, tasks take longer to complete than anticipated. This is due largely to 
underestimating the work involved, competing priorities, and intruding activities that 
are outside of the control of the person charged with completing the assignment. The 
distinction between time needed and time available is also an issue. Whereas a task 
may require a week of work, the elapsed time could be spread over a number of weeks. 
The longer the elapsed time, the more likely that other priorities interfere with completion. 
Keeping people motivated and on task is difficult and requires considerable organizational 
and negotiation skills. Coordinators who were assigned exclusively to their EQI project 
were more successful in this regard than those who divided their time among other 
responsibilities. 

Timeliness is an important criterion in the value and usefulness of information. The 
semiannual meetings required project updates and the annual published status reports were 
inducements to adhere to the project timelines. All opportunities to hone writing skills 
helped project coordinators complete their reports. Documentation is demanding and very 
time-consuming, yet essential for others washing to replicate a particular project. 

A number of timing issues remain. For example, should students be sampled or 
should the results be based on a census? Which outcomes should be measured annually 
and which periodically? Collecting information about the conditions (context, inputs, 
processes) may not disrupt instructional time, but can exact a toll in terms of collecting, 
analyzing and reporting results. Deploying more staff to complete these tasks can reduce 
elapsed time but may result in less accuracy because of the greater number of people 
working on different parts. Competing demands of accuracy and timeliness become a 
major issue. 
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Recommendations 

The following recommendations summarize th^ lessons from the EQI initiative regarding 
how to maximize the impact of large-scale research projects. 



Table 3 

Recommendations for Maximizing the Impact of Large-Scale Projects 



1. Leadership 



2. Scope 



3* Collaboration 



4. Communication 



5. Staff 



6. Finances 



7. Information 

8. Support 

9. Time 

10. Celebration 



Strong leadership is required to introduce an innovation. Leadership should be 
shared among all participants for maximum impact Working with change agents 
directly rather than through an intermediary organization improves effectiveness 
and efficiency. 

A broad conceptual framework which permits selection of specific components 
for intensive investigation encourages creativity. It also makes the work load 
manageable. Several projects focusing on different aspects of a framework 
contribute significantly to the understanding of the whole. 

This is the preferred modus operandi because it invites partiC! pation and develops 
ownership and commitment. All who have an interest in the project should be 
involved - students, teachers, administrators, parents, trustees, stakeholder 
groups, government officials, and the public. 

This is an essential ingredient for successful implementation. There should be 
a variety of communication strategies and media - both formal and informal 
verbal, print and multimedia. 

Staff who have an interest in, commitment to and the necessary expertise should 
be invited to take part. Voluntary participation promotes success. Motivation 
becomes essential as the tasks become more challenging. 

Adequate financial resources should be provided to complete a project on time. 
Resources in kind, such as releasing staff members from current obligations to 
undertake new tasks, reassigning priorities for existing staff and financial 
commitments, and eliminating redundant or unproductive activities, provide 
alternatives where money is unavailable. 

Informed discussion requires information. Experts can help identify sources and 
provide advice. Available information from the literature and other sources should 
form the basis of the work. This resource helps to focus on the important aspects 
of the project. 

This is essential at all levels - government, trustees, superintendent, principal, 
teachers, students, parents, the public. 

This resource cannot be underestimated. Ensure adequate time by organizing 
the work, assigning adequate resources to ensure completion, and milestones 
to encourage adherence to deadlines. 

Recognize stages of achievement in a variety of ways and celebrate the 
completion to honor participants and to encourage continuation and adoption. 
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Provincial Implementation 

Alberta Education is committed to implementing a set of provincial indicators. The EQI 
initiative is one of the projects designed to promote open information to help stakeholders 
address issues and realistically set improvement targets. Other provincial initiatives 
focusing on reporting results include the Vision for the nineties ... a plan of action (1991), 
which established priority directions, and its annual progress report entitled Achieving the 
vision The new province-wide electronic Student Information System will include all 
students from Early Childhood Services to grade 12 by the fall of 1995; this will facilitate 
analysis and tracking of student results over time. 

The three-year EQI projects can be considered as Phase I of a reform strategy that 
may well take a decade to implement fully. The partnership identified ten indicators 
that participants felt could comprise a set of provincial indicators which encompasses 
the cognitive, affective and behavioral domains measured in multiple ways by different 
people. This broad range of outcomes provides a better picture of the desired outcomes 
of schooling. Asking parents, teachers, and the public how well students are achieving 
selected outcomes helps to confirm the results. It can also help to improve student 
learning by involving stakeholders in the educational enterprise. 

The next steps; involve promoting the outcomes of the EQI initiative. The project 
teams are using the materials they have developed; they are continuing their collaboration 
by incorporating materials and strategies from the other projects in their districts as well. 
Many friendships developed among partners over the three years which will ensure 
continuing dialogue on indicators and ways to improve education. Teams are also 
presenting their materials to teachers and administrators at professional development 
days, teachers' conventions and conferences. 

Alberta Education is promoting and extending the work of the EQI initiative by 
publishing the reports and handbooks, keeping it on the provincial agenda, establishing 
a Provincial Advisory Committee, and expanding the EQI network to involve other 
interested school jurisdictions. The twelve participating districts represent 8% of the 
143 districts in Alberta. The department intends to continue discussion with all Alberta 
school jurisdictions on which common indicators are important to report. Results on this set 
will then be published and made available to the public by all districts through their Annual 
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Education Report. Common indicators are essential for comparing results and for 
interpreting them in light of the local context and decisions regarding the allocation of 
resources. Districts will continue to publish other information they feel is important to 
their local community as well. 

Quality is high on the public agenda in Canada. On a national survey of the public's 
views and attitudes on education, one in five respondents (18%) mentioned the overall 
quality of education as a critical issue, with another 10% suggesting that more attention 
should be focused on M the basics" and 7% urging reform (Angus Reid, 1993, p. 1 1). This 
poll of 1,504 adult Canadians found that 46% felt that the quality of education was worse 
today than it had been 25 years earlier, this assessment has deteriorated since the 1986 poll 
which found 36% felt that education was worse than a quarter century earlier (ibid, pp. 17- 
18). Public perceptions are often formed on the basis of opinion and media coverage, and in 
the absence of solid data reporting trends over time. Educational organizations can help to 
focus the debate by providing sound information from multiple sources. Indicators that are 
agreed upon by stakeholders and tracked over time to establish trends are a way to inform 
the public of the outcomes of education. 

Conclusions 

The EQI initiative has had a positive impact on education in Alberta. The twelve participating 
districts have incorporated concepts, strategies and materials into their operations. This 
has resulted in changed policies, refocused priorities, and improved instructional and 
organizational practices. Alberta superintendents have embraced the idea of indicator 
systems and many are using the EQI materials. The department is also incorporating ideas 
and strategies into its information system. We have essentially completed Phase I of a 
provincial reform strategy to improve education by enhancing information for better 
planning, policies, practices and decisions. There has been much interest in EQI with 
many requests for Achieving Quality. 

What have we learned from the Educational Quality Indicators initiative? We 
learned that a large-scale reform initiative focused on producing better information about 
education can help to improve education. A rich diversity of approaches to developing 
and implementing indicator systems within the framework of a multidimensional model 
of education resulted in many innovative ways to define important outcomes of schooling 
and ways to measure them. 
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There is no single ideal indicator system nor is such a tool a panacea for education. 
A useful indicator system, however, focuses on student outcomes, is based on important 
goals of schooling and desirable personal characteristics, represents a balance among 
cognitive, affective and behavioral outcomes, and includes multiple measures, levels of 
analysis and perspectives. Educational indicators can provide better information to help 
improve education. They can provide a more balanced picture of the operation and 
outcomes of schooling, particularly if they focus on a broader range of desired outcomes. 
By focusing on student learning, and interpreting and reflecting on the findings, better 
decisions about schooling will result. 

The collaborative quest underscored the complexity of education and generated 
creative solutions to some of the issues facing education. Collaboration, communication 
and commitment were the hallmarks of effective implementation. 
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